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ROAD LANE MARKER RECOGNITION 

BACKGROUND OF THE INVENTION 

The present invention relates to technique for 
recognizing a road lane nnarker, such as a white line, 
5 ahead of a vehicle for automatic steering systenns and 
various other applications. 

Published Japanese Patent Application 6-20189 and 
Published Japanese Patent Application 8-261756 disclose 
road lane recognizing devices using forward road image 
10 ahead, 

SUMMARY OF THE INVENTION 

However, there are objects, such as water in a rut, 
guard rail, and dual white lines, which can be mistaken for 
a lane marker. Such an object often extends along an 
15 actual lane marker, and binary image of the object tends 
to lead the road geometry estimation into error. 

It is therefore an object of the present invention to 
provide road lane marker recognition apparatus and/or 
method capable of estimating road geometry accurately 
20 without misrecognition. 

According to the present invention, a road lane 
marker recognition apparatus for obtaining a forward road 
image of a road ahead of a vehicle with an imaging device 
and estimating road parameters of a road model equation, 
25 comprises: 

a window setting section to set, in the road image, a 
plurality of windows along a road lane marker on the road 
so that the road lane marker is contained in each window; 
a candidate lane marker calculating section to 
30 calculate a candidate lane marker which is likely to be the 
road lane marker in each window; 
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a position accuracy judging section to judge a 
position accuracy in position of tlie candidate lane marker 
witli respect to the road lane marker in eacli window; and 
a road parameter estimating section to estimate the 
5 road parameters of the road model equation representing 
a road shape of the road and vehicle attitude with respect 
to the road, the road parameter estimating section being 
configured to estimate the road parameters inclusive of a 
vehicle lateral displacement, by using at least a gradient 

10 of the candidate lane marker for one window judged to be 
lower in the position accuracy, and by using an image 
coordinate of the candidate lane marker for one window 
judged to be higher in the position accuracy. 

According to the present invention, a road lane 

15 marker recognition process comprises; obtaining a forward 
road image of a road ahead of a vehicle; setting, in the 
road image, a series of windows arranged along a road 
lane marker on the road; calculating a candidate lane 
marker which is likely to be the road lane marker in each 

20 window; judging a position accuracy in position of the 
candidate lane marker in each window; and estimating 
road parameters of a road model equation inclusive of a 
vehicle lateral displacement, by using at least a gradient 
of the candidate lane marker for one window judged to be 

25 lower in the position accuracy, and by using an image 
coordinate of the candidate lane marker for one window 
judged to be higher in the position accuracy. 

The other objects and features of this invention will 
become understood from the following description with 

30 reference to the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a schematic view showing a road lane 
marker recognition apparatus according to a first 
embodiment of the present invention. 
5 FIG. 2 is a schematic view showing a vehicle 

equipped with the road lane marker recognition apparatus. 

FIG. 3 is a flowchart of an image processing program 
according to the first embodiment. 

FIG. 4 is a flowchart of a road parameter estimating 
10 subroutine in the image processing program of FIG. 3. 

FIG. 5 is a view showing a plurality of windows used 
in the road lane marker recognition apparatus according 
to the first embodiment. 

FIG. 5 is a view showing candidate white line points 
15 (or candidate lane marker points) in one window according 
to the first embodiment. 

FIG. 7 is a view showing possible candidate white 
lines (or candidate lane markers) in one window according 
to the first embodiment. 
20 FIG. 8 is a table showing an array by Hough 

transform for searching a candidate white line (or 
candidate lane marker) according to the first embodiment. 

FIGS. 9A and 9B are tables for illustrating Hough 
transform in terms of array elements used in the first 
25 embodiment of the present invention. 

FIG. 10 is a graph for illustrating a peak frequency 
when there exists a pseudo white line (or pseudo lane 
marker) obtained by Hough transform in the first 
embodiment. 

30 FIG. 11 is a graph for illustrating a peak frequency 

when there exists no pseudo white line obtained by Hough 
transform in the first embodiment. 
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FIG. 12 is a view showing road coordinates of an 
intersection between a candidate winite line and an upper 
side of a window set in the first embodiment. 

FIG. 13 is a schematic view showing a road lane 
5 marker recognition apparatus according to a second 
embodiment of the present invention. 

FIG. 14 is a flowchart of an image processing 
program according to the second embodiment. 

FIG. 15 is a view for illustrating faraway windows 
10 which are judged lower in accuracy in the second 
embodiment. 

FIG. 16 is a perspective view illustrating a road 
formed with a rut along a road white line so that a pseudo 
white line can be produced. 
15 FIG. 17 is a perspective view illustrating a road with 

a guard rail which can cause a pseudo white line. 
nETAILED DESCRIPTION OF THE INVENTION 

Preferred embodiments of the invention will now 
be described in detail with reference to the 
20 accompanying drawings. 

Fig. 2 is a schematic view of a vehicle 12 equipped 
with a road lane marker (white line) recognition 
apparatus 10 according to an embodiment of the 
invention, for recognizing a road white line (or lane 
25 marker). In Fig. 2, vehicle 12 is provided with an image 
acquisition device (or imaging device) 14 for acquiring 
an image of a road in front of the vehicle and a 
processor 16 for processing an image signal (hereinafter 
simply referred to as "image") acquired by image 
30 acquisition device 14. In this example, image 

acquisition device 14 is a CCD camera capable of picking 
up images with a high sensitivity, and image acquisition 
device (hereinafter referred to as "camera") 14 is 
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mounted to compartment ceiling 12a in the middle of a 
front part thereof such that it is directed forward and 
downward to acquire an image of road R in front of 
vehicle 12 through front glass 12b. Processor 16 is 
5 provided in an appropriate location where it is not 
affected by heat from vehicle 12 and wind and rain to 
perform control such that road white line 18 on the road 
is recognized from the image of road R acquired by 
camera 14 to estimate the shape of the road and the 

10 attitude of the vehicle relative to the road. 

Fig. 1 is a diagram showing a basic configuration 
of the lane marker recognition apparatus 10 according to 
this embodiment of the invention. In Fig. 1, road lane 
marker recognition apparatus 10 has a picture memory 

15 (or image memory) 15 for storing an image of a road in 
front of the vehicle acquired by image acquisition device 
14 mounted on the vehicle, and processor 16, 
constituted by, in this example, a microprocessor, for 
estimating road parameters based on the road image 

20 stored in the image memory 15. 

Processor 16 has a window setting section 101 for 
providing a plurality of windows on the road Image 
stored in the image memory 15 in the direction in which 
a white line (or lane marker) on the road extends such 

25 that the white line on the road is included in each of the 
windows; a white line candidate point extracting section 
(or lane marker candidate point extracting section) 102 
for processing the image in each of the windows to 
extract candidate points for a white line (or lane marker; 

30 a candidate white line calculating section (or candidate 
lane marker calculating section) 103 for calculating the 
image coordinates and gradient of a first candidate white 
line having the highest probability of being a white line 
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based on the white line candidate points extracted in 
each of the windows; a pseudo white line (or pseudo 
lane nnarker) presence judging section (white line 
position accuracy judging section) 104 for judging 
5 whether a second candidate white line resembling to the 
first candidate white line exists or not, that is whether a 
pseudo white line (or pseudo lane marker) exists or not 
in each of the windows; and a road parameter estimation 
section 105 for estimating road parameters including at 

10 least lateral displacement of the vehicle relative to the 
lane, by using the image coordinates and gradient of the 
first candidate white line calculated in each of the 
windows and a road model equation. The sections 101 
through 105 are achieved by a program control in 

15 processor 16. 

Road parameter estimation section 105 estimates 
the road parameters including the vehicle lateral 
displacement relative to the lane by using the image 
coordinates and gradient of the first candidate white line 

20 for each of windows which are judged as having a 

pseudo white line (or as lower in position accuracy) and 
by using the image coordinates of the first candidate 
white line for each of windows which are judged as 
having no pseudo white line (or as higher in position 

25 accuracy). 

Road parameter estimation section 105 of this 
example uses Equation 1 shown below as a road model 
equation to estimate the shape of the road and the 
attitude of the vehicle relative to the road. 

30 [Equation 1] 
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A process for deriving Equation 1 for such 
modeling of a road is detailed in ''Investigation on 
Autonnatic Path Tracking Using Extended Kalman Filter" 
(20005494) proposed by the inventor of the present 
5 application at the 2000 autumn meeting of Society of 
Automotive Engineers of Japan (October 19, 2000). The 
entire contents of this document are hereby incorporated 
by reference. 

Road parameters A, C, and H in Equation 1 

10 are lateral displacement of the vehicle relative to the 

lane (A), the curvature of the road (B), the yaw angle of 
the vehicle with respect to the lane (C), the pitch angle 
of the vehicle (D), and the height of the image 
acquisition device from the road surface (H), 

15 EO is a constant which represents the width of the 

lane (the distance between inner sides of left and right 
white lines on the left and right sides of the lane); f is a 
penetration conversion constant of the camera; j is a 
parameter for distinguishing between the left and right 

20 white lines which is 0 for the left white line (] = 0) and 1 
for the right white line (j = l). Coordinates (x, y) 
represent the location of an arbitrary point on the inner 
edge of the left or right white line on the image, the 
origin being the upper left corner of the image, the 

25 rightward direction being the positive direction of the x- 
axis, and the downward direction being the positive 
direction of the y-axis. 

Thus, Equation 1 represents a road model. 
However, various modifications. For example, EO may 

30 be a variable; and H may be fixed. The lane width may 
be the distance between the centers of the left and right 
white lines; and (x, y) may be defined as the image 
coordinates at the middle of the white line width. 
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Figs. 3 and 4 show a program for estimating road 
parameters with the processor 16. In this example^ the 
process is repeated at regular time intervals of 30 msec 
during running operation of the vehicle. 
5 In the process in Fig. 3, first, a forward image of a 

road ahead the vehicle is taken from camera 14 to image 
memory 15 (step SI). Then, a plurality of (m) windows 
are set on the acquired image (step S2). The quantity 
m is the number of windows. A window processing 

10 number i is set at an initial value of l(step S3), 

Changes in luminance in the horizontal direction are 
detected from image data of the i-th window (hereinafter 
referred to as "window i'') 20 to extract white line 
candidate points 21 for recognizing the position of an 

15 edge of a white line 18 (step S4), Then, a first 

candidate white line 22 having the highest probability of 
being a white line is calculated from those white line 
candidate points, based on the coordinate positions of 
the white line candidate points 21 in the window (step 

20 S5). 

Pseudo white line presence judgment is then made 
to see whether there is a second candidate white line 
having a gradient similar to that of the first candidate 
white line in the window (step S6). If there is a second 

25 candidate white line according to the result of the 

judgment at step S6, a flag WEi for the window is set at 
1 (step S7). If there no second candidate white line 
exists, flag WEi for the window is set at 0 (step SB). 
Then, window number i is incremented by 1 (step S9), 

30 and it is judged whether i has exceeded the number of 
the windows (m) (step SIO). If not, the program 
returns to step S4 to process the next window. In this 
way, all the windows are subjected to preprocess or 
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preparatory process. When all the windows are finished 
and hence i has exceeded the number of the windows 
(nn), the program returns to the first step of acquiring 
an image in front of the vehicle (step SI) after 
5 performing a road parameter estimation process (step 
S20). 

The processes of the above program will be 
described in detail on a step-by-step basis. 
(1) Acquisition of forward image 
10 At regular time intervals of the predetermined 

3 time length for repeating the program of FIG. 3, an 

!2 image signal of a forward image in front of the vehicle 

y formed by camera 14 is taken into image memory 15, 

The image is represented as luminance data of each 
Q 15 pixel. 

Q (2) Setting of a plurality of windows 

^ A plurality of windows 20 are arranged along a 

\i road white line (or each of left and right white lines), as 

fj shown in Fig. 5 (five windows for each white line on the 

20 left or right sides, in the example shown in Fig. 5). By 
using values of the parameters of the road model 
obtained in the previous execution of the program, the 
position of each window 20 is so determined that a white 
line 18 on the road is included in each of the windows 20, 
25 so that a white line 18 can be detected in each window 
20. The vertical direction in Fig. 5 represents the 
direction of the y-axis of the image coordinate system. 
The positive direction of y extends downward in Fig. 5. 
The horizontal direction in Fig. 5 represents the direction 
30 of the X-axis of the image coordinates. The positive 
direction of x extends rightward in Fig. 5. 

Let us assume here that the y-coordinate of the 
upper side 20a of the n-th window (i = n) is represented 
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by yn and that the result of the previous parameter 
estimation is {A(-l), B(-l), C(-l); D(-l), H(-l)}. Then, 
from Equation 1, the coordinate xest of the lateral 
middle of the current window 20 along the lateral 
direction between the left and right ends is expressed by 
Equation 2, 
[Equation 2] 



xest = 



The width of the window 20 in the x-direction may 
be a fixed value, or alternatively the window width may 
be determined from the variance of the parameters as 
disclosed in Published Japanese Application Publication 
15 {Kokai) No. 8( 1995)-251756. 

(3) Initialization of window number i 
Loop control parameter i to ensure the preparatory 
process of steps S4 to S9 on all windows 20 is initialized 
to 1. 

20 (4) Extraction of white line candidate points 

One or more white line candidate points are 
chosen from the image in the range of each window set 
in the image acquired by camera 14, by using an edge 
detecting (luminance change detecting) process such as 

25 a Sobel filtering process. In this edge detecting process, 
when the luminance of a left-hand pixel (with a smaller 
x-value in the image coordinates) is greater than the 
luminance of a right-hand pixel (with a greater x-value 
in the image coordinates), then the filter output is 

30 positive as shown in Fig. 6. In this example, the 

position of the inner side of a white line 18 having a 
certain width is regarded as the position of a white line 
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18. Accordingly, a point at which the filter output 
exceeds a positive discrimination value is regarded a 
white line candidate point in the case of the white line 
on the left, and a point at which the filter output is less 
5 than a negative discrimination value is regarded a white 
line candidate point in the case of the white line on the 
right. The entire image data in window 20 are scanned 
to extract white line candidate points by the filter output 
using the above-described discrimination values. The 
10 above-described discrimination values may be 
D appropriately set depending on the average luminance, 

% contrast, and so on of the entire image or each window, 

P (5) Calculation of candidate white line 

i .V = 

Candidate white line 22 is searched from the set of 

ess;; 

£ 15 white line candidate points 21 extracted from window 20. 
r Hough transformation or the least squares method may 

J;J be used for the search. This embodiment employs 

HI Hough transformation. In Hough transform straight line 

^ approximation, a straight line passing through the 

H 20 greatest number of white line candidate points 21 among 
straight lines passing through window 20 as shown in Fig. 
7 is treated as a first candidate white line having the 
highest probability of being a white line. 

A straight line passing through a coordinate point 
25 (x, y) can be expressed by the following Equation 3 
using parameters a and b where a represents the 
gradient (ax/ay) of the candidate white line, and b 
represents an x-intercept of the candidate white line. 
[Equation 3] 
30 X = ay + b 

Specifically, when a is determined and the 
coordinate of a candidate white point is given by (x, y), 
then it is possible to calculate the parameter b by using 
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Equation 3, and thereby to obtain an array according to 
Hough transform as shown in Fig. 8. The table shows 
values of the paranneter a with constant increments in 
the horizontal direction and shows values of the 
parameter b with constant increments in the vertical 
direction. The blank cells of the array represent zero. A 
true value is included in combinations (a, b) for which 
"1" is set. Such an array is created for every white line 
candidate point. The arrays thus created are 
superposed, and the values of each array element are 
totaled to obtain an array shown in Fig. 9A. The value 
zr of an array element (ar, br) in Fig. 9A represents the 
number of white line candidate points penetrated by a 
line of Equation 4. 
[Equation 4] 

X = a^y + br 

Therefore, a candidate white line passing through 
most white line candidate points is a first candidate 
white line having the highest probability that a 
candidate is a true white line, and the determination of a 
and b for that candidate white line amounts to the 
calculation of the first candidate white line. Namely, an 
array element (a, b) having a maximum array element 
value z represents a first candidate white line. 

When the candidate white line is thus calculated, 
then a road coordinate pair (xn, yn) is calculated. The 
road coordinates (xn, yn) are the coordinates of an 
intersection between the first candidate white line 22 
and the window upper side 20a in the window No. n, as 
shown in Fig. 12. The y-coordinate yn of upper side 20a 
of window 20 is a value fixed for the window No. n. The 
x-component xn of the road coordinates is calculated 
using Equation 5. 
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[Equation 5] 

Xn = anYn + bp 

(6) Judgment of presence of pseudo white line 
Judgment on presence of pseudo white line is 
5 made based on the array shown in Fig. 9A. This 

judgment is to examine whether any second candidate 
white line having a gradient approximately equal to that 
of the first candidate white line (,so that a gradient 
difference between the gradients of both lines is equal to 

10 or smaller than a predetermined value) exists in the 
image in the window, to see whether any pseudo white 
line exists in the window. When, for example, a pseudo 
white line, such as a white guide rail or a region of high 
luminance generated by reflection from water collected 

15 in a wheel track is included in the image in the window, 
edges of such a pseudo white line in the horizontal 
directions are also extracted as white line candidate 
points as a result of the above-described edge detection. 
Therefore, the white line candidate points may include 

20 true white line candidate points indicating the right or 
left edge of a white line and false white line candidate 
points originating from the pseudo white line. 

In addition to a first candidate white line that is a 
candidate white line having the highest probability of a 

25 true white line, a second candidate white line having a 
gradient similar to that of the first candidate white line 
is calculated from image data of a window including one 
or more pseudo white lines. 

Depending on the conditions of the road, a 

30 candidate white line calculated from white line candidate 
points originating from a pseudo white line can be 
chosen as a first candidate white line having the highest 
probability of being the white line instead of a true white 
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line. In such a case, the presence of any pseudo white 
line is checked in order to prevent introduction of error 
in the estimation of road parameters even if a pseudo 
white line is erroneously recognized as a first candidate 
white line. In estimating road parameters as described 
later, an equation including road parameters other than 
a vehicle lateral displacement relative to the lane is 
used for a window which is judged as having a pseudo 
white line. 

Referring to the parameter a indicating the 
gradients of candidate white lines in the array in Fig. 9A, 
the values in each column are totaled, and the maximum 
sum is chosen. If a plurality of candidate white lines 
having a gradient difference equal to or smaller than a 
predetermined value on the image are found in this 
process, one of such candidates is selected. The 
predetermined value is set because a true white line and 
a pseudo white line in parallel therewith are not parallel 
on the image, and the value is determined by the 
maximum value of the distance between the true white 
line on the road and the pseudo white line to which the 
process is to be applied and by the camera height H. 

The selected value of the parameter a is 
represented by a|M, and the variance of b (x-intercepts) 
in the column of the value a^ is calculated. If the 
predetermined value is greater than the interval of the 
parameter a (e.g., lap - ap-il), when the values in each 
column is totaled, the maximum sum is selected by 
obtaining sums, including adjacent columns (e.g., aN-i 
~ aiM + i) until the range of a exceeds the predetermined 
value as shown in Fig. 9B, and the variance of b is 
calculated in the columns of a^. 
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If the variance of b is equal to or smaller than a 
predeternnined value, it is judged that no second 
candidate white line having a gradient similar to that of 
the first candidate white line or no pseudo white line 
5 exists. Otherwise, it is judged that a pseudo white line 
exists. If a pseudo white line exists, a plurality of peaks 
appear as shown in Fig. 10, resulting in greater variance. 
A set of windows n judged as having a pseudo white line 
are represented by Ga, and a set of windows n having no 

10 pseudo white line are represented by Gb. 

Then, as shown in Fig. 11, the value of b 
corresponding to the greatest one in the column of apj 
(bn in Fig. 11) is selected. Thus, a pair of a and b which 
defines a first candidate white line having the highest 

15 probability of being a white line is selected. At this time, 
a pair of a and b defining a pseudo white line instead of 
a true white line may be erroneously selected as a first 
candidate white line if white line 18 on the road is 
partially faded, or there is a white guard rail, or light is 

20 continuously reflected by water collected in a rut, as 
shown in FIGS. 15 and 16. 

(7) Setting of judgment flag indicating the 
presence of pseudo white line 

Flag WEi of a window is set at ''1" when it is 
25 judged as having a pseudo white line as a result of 

judgment on the presence of a pseudo white line at step 
S7, and flag WEi of a window is set at "0'' when it is 
judged as having no pseudo white line. This flag is 
referred to during the estimation of road parameters to 
30 be described later. 

(8) Estimation of road parameters 
Equation 1 is differentiated by y to obtain 

Equation 6. 
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[Equation 6] 

dx f ~A + jE^ \ B*H* f" 

Equation 7 is obtained by eliminating A from 
Equations 1 and 5, 
[Equation 7] 



At this time, A represents a parameter of the 
lateral position of the vehicle relative to the lane, and 
EO representing the lane width is eliminated at the same 
time when A is eliminated. That is, Equation 7 is 
satisfied for any road width. In other words. Equation 7 
is satisfied by any line segment which is in parallel with 
white line 18 on the road without using true white line 
18 on the road. Specifically, in practice, a pseudo white 
line originating from a guide rail, reflection of light from 
water collected in a rut, a double white line, or the like 
is frequently in parallel with a true white line 18 on the 
road. When such a pseudo white line Is detected, road 
parameters excluding A (parameters B, C, D, and H) can 
be accurately estimated from the gradient of the pseudo 
white line (ax/ay), the road coordinates (x, y), and 
Equation 7 using a technique such as the least squares 
method or Kalman filter. Parameter A may be estimated 
from the data of a window in which no pseudo white line 
exists along with other parameters. 

When Kalman filter is used for the estimation of 
the parameters, a parameter z can be estimated from an 
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equation expressed as Equation 8 below. A value of 
parameter yy and the structure of f in the same equation 
are known, and yy is in the form of a vector in general, 
[Equation 8] 
5 yy = f(2) 

A description will now be made on derivation of an 
observation equation (so called) corresponding to 
Equation 8 in the case wherein a pseudo white line 
exists and the case wherein no pseudo white line exists. 
10 (1) When a pseudo white line exists, Equation 7 is 

used, and Equation 9 is satisfied because the left side of 
Equation 7 represents a gradient on an image. 
[Equation 9] 



15 a„ = 



nGGa 



Equation 9 can be rewritten as follows, 
ax/ay = fw (B, C, D, H) Equation 9' 

(2) When no pseudo white line exists, the above- 
20 mentioned Equation 1 holds as Equation 10. 
[Equation 10] 

neCb 

25 Equation 10 can be rewritten as follows. 

X = fz (A, B, C, D, H) - Equation 10' 
In general, Equations 9' and 10' are constituted by 
a plurality of colunnns. These are subjected to linear 
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approximation and arranged to 
shown below. 
[Equation 11] 
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In Determinant 11, m indicates the number of 
windows set in one screen. When window No. n belongs 
to Ga (a set of windows having therein a pseudo white 
10 line), zn = (ax/ay)n = an, and Mnl is zero. When 

window No. n belongs to Gb (a set of windows having 
therein no pseudo white line), zn = xn. (Each of the 
sets Ga and Gb is a subset of a set of the windows m in 
number. ) 

15 Subroutine P3 for the process of estimating road 

parameters will now be described with reference to FIG. 
4. 

First, window number parameter i is initialized to 
1 (i = l) at step S21. Then, the value of flag WEi 

20 associated with window No. i is checked at step S22 to 
branch to a process on a window in which a pseudo 
white line exists or a process on a window in which no 
pseudo white line exists. 

In the case of a window in which no pseudo white 

25 line exists, at step S23, zi in Equation 11 is set at xi, 
and the right side of Equation 10 is linearly 
approximated to respectively set Mil, Mi2, Mi3, Mi4, and 
Mi5 at dfz/dA, dfz/dB, dfz/dC, dfz/dD, and dfz/dH at 
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y = yi. In FIG. 4, {Q | x = xi, y = yi} represents Q in the 
case that x is equal to xi and y is equal to yi. 

In the case of a window in which a pseudo white 
line exists, at step S24, zi in Determinant 11 is set at 
5 (dx/dy)i, and the right side of Equation 9 is linearly 

approximated to respectively set Mil at 0, and Mi2, Mi3, 
Mi4, and Mi5 at dfw/dB, dfw/dC, dfw/dD, and dfw/dH for 
which the values of x and y are (xi, yi). 

Then, the window number i is Incremented by 1 at 

10 step S25, and it is judged at step S26 whether window 
No. i has exceeded the number of windows m. If the 
number of windows m has not been exceeded, the 
program returns to step S22 to process the next window. 
If the number of windows m has been exceeded, 

15 parameters (A, B, C, D, H) are estimated by the Kalman 
filter process at step S27, and then the process returns 
to the main routine. 

Fig. 13 is shows a configuration of a second 
embodiment of a road white line recognition apparatus 

20 according to the invention. In Fig. 13, a road white line 
recognition apparatus 10a includes an image memory 15 
for storing an image of a road in front of a vehicle 
acquired by an imaging device 14 loaded on the vehicle, 
and a processor 16 for estimating road parameters 

25 based on the image of the road stored in image memory 
15, as in the preceding embodiment. 

Processor 16 has a window setting section 101 for 
providing a plurality of windows on the road image 
stored in image memory 15 in the direction in which a 

30 white line on the road extends such that the white line 
on the road is included in each of the windows, a white 
line candidate point extracting section 102 for 
processing the image in each of the windows to extract 
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candidate points for the white line, a candidate white 
line calculating section 103 for calculating the 
coordinates and gradient of an image of a candidate 
white line having the highest probability of being the 
5 white line based on the white line candidate points 
extracted in each of the windows, a window position 
judging section (white line position accuracy judging 
means) 110 forjudging the distance between the road 
surface in each of the window and the vehicle, and a 
10 road parameter estimation section 105 for estimating 
5 road parameters including at least lateral displacement 

^ of the vehicle relative to the lane using the image 

y coordinates and gradient of the image of the candidate 

% white line calculated in each of the windows and a road 

O 15 model equation. 

Q Road parameter estimation section 105 estimates 

the road parameters including the vehicle lateral 
displacement of the vehicle relative to the lane by using 
y the image coordinates and gradient of the candidate 

20 white line for a window in which the road surface is 

judged to be located far from the vehicle, and by using 
the image coordinates of the candidate white line for a 
window in which the road surface is judged to be not far 
from the vehicle. 

25 The road model equation and road parameters in 

the second embodiment are substantially identical to 
those used in the first embodiment. 

While operations in the program P2 in the present 
embodiment shown in FIG. 14 are substantially identical 

30 to those of program PI executed in the flowchart in 

FIG. 3 described in the first embodiment, in the present 
embodiment, an observation equation selecting 
operation is performed at step S12 instead of step S6 of 
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program PI in the first embodiment^ and the operation 
at step S12 will be described below. Operations in the 
flowchart in FIG. 14 identical to those in the flowchart in 
FIG. 3 are indicated by like step numbers and will not be 
5 described repetitively. 

Selection of observation equation 
Group Ga or Gb Is selected for a window having a 
window number n as in the first embodiment. For group 
Ga, the observation equation is given by Equation 9 

10 (Equation 9'). In the case of group Gb, the observation 
equation Is expressed by Equation 10 (Equation 10'). In 
the first embodiment, Ga is selected when it is judged 
that a pseudo white line exists. In the second 
embodiment, Ga is selected not only when it is judged 

15 that a pseudo white line exists but also when a 

candidate white line 22 is judged in a faraway window 
20a remote from the vehicle. In this example, the 
faraway window is the uppermost window 20a shown by 
a bold line in FIG. 15. It is optional, however, to regard 

20 a plurality of consecutive windows in an upper part 

including the uppermost window as faraway windows to 
be included in the group Ga. 

According to the second embodiment, a gain G of 
parameter A with respect to a change in xn can be 

25 obtained as Equation 12 from Equation 1. 
[Equation 12] 



the vertical axis, and the value of yn decreases with the 
distance from the vehicle. Since yn appears in the 
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denominator of G, G increases with increase in the 
distance fronn the vehicle. This nneans that an error on 
the image results in greater Influence in the estimation 
of parameter A for window 20 which is located further 
5 from the vehicle. In practice, since the resolution of an 
image is unchanged regardless of the distance, the 
accuracy of estimation of parameter A is lower for 
window 20 for seeing a greater distance. Therefore, the 
accuracy of estimation of a road model can be improved 

10 by not estimating parameter A in window 20a for a great 
distance for which reduction in the estimating accuracy 
of parameter A is predicted. 

In each of the above embodiments, candidate 
white line 22 in window 20 is treated as a straight line 

15 for approximation, and such a straight line 

approximation makes it possible to increase the speed of 
the process. Specifically, if candidate white line 22 is 
approximated to a high-order curve, the approximating 
process itself requires an enormous time for calculation, 

20 and a process for calculating gradients on the image will 
be required. While a gradient must be re-calculated by 
specifying the coordinates in the case of a curve, the 
gradient of a straight line is determined during the 
process of approximation. Therefore, the size of 

25 windows 20 may be set such that white line 18 on a road 
can be approximated to a straight line in each of 
windows 20 even when white line 18 is a curved line. 

Since Hough transformation is used in 
approximating candidate white line 22 to a straight line, 

30 when a plurality of candidate white lines 22 are acquired, 
there is no need for re-calculating the gradients of the 
plurality of candidate white lines on the image, which 
makes it possible to increase the speed of the process. 
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This is advantageous In a system for which high speed 
processing is required such as a vehicle traveling at a 
high speed. 

FIG. 16 shows a road 1 formed with a rut 2 in 
5 which water is collected. Sunlight or streetlight can 
cause reflection from the surface of a pool (or water 
film) in dependence on the incident angle, and appear, 
in an electronic image, as a high luminance line 
resembling a road white line 3. FIG. 17 shows a white 

10 guardrail 5 and a white line 3 extending in the shade 6 
of the guardrail 5. In this case, the luminance of the 
white line in the shade 6 is decreased, and the white 
guardrail 5 and an unshaded part of the white line 3 
could cause misrecognition, as a road model, of a line L 

15 connecting a detection point ml on the unshaded white 
line segment and a pseudo detection point m2 on the 
guardrail 5. The white line recognition system according 
to the present invention can avoid such misrecognition 
by treating each window or the window of the subset Ga 

20 in a first mode (S24) and treating each window or the 
window of the subset Gb in a second mode (S23). 

This application is based on prior Japanese Patent 
Application No. 2000-301453 filed on September 29, 2000, 
and prior Japanese Patent Application No. 2001-179204 

25 filed on June 13, 2001. The entire contents of these 
Japanese Applications are hereby incorporated by 
reference. 

Although the invention has been described above by 
reference to certain embodiments of the invention, the 
30 invention is not limited to the embodiments described 
above. Modifications and variations of the embodiments 
described above will occur to those skilled in the art in 
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light of the above teachings. The scope of the invention is 
defined with reference to the following claims. 
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